[AMD] feat: MiniMax M3 day-zero benchmark for MI300X#1746
Conversation
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
…x-dayzero # Conflicts: # .github/configs/amd-master.yaml # perf-changelog.yaml
cquil11
changed the title
cquil11
requested review from
1am9trash,
billishyahao,
chunfangamd,
seungrokj and
yctseng0211
as code owners
|
Claude finished @cquil11's task in 1m 30s —— View job Reviewing PR #1746
ReviewLGTM — no blocking issues found. Config ( Benchmark script ( Launch script ( Perf changelog: New entry is correctly appended at the end of the file. |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27482347617 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27482907707 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27485974465 |
|
/reuse-sweep-run |
1 participant
Summary
minimaxm3-fp8-mi300x-vllmusing the dedicatedvllm/vllm-openai-rocm:minimax-m3image already used by MI355Xchi-mi300x-049)/dev/kfdand/dev/driinto the rootless Pyxis/Enroot containerValidation
bash -n benchmarks/single_node/fixed_seq_len/minimaxm3_fp8_mi300x.sh runners/launch_mi300x-amds.shpython -m pytest utils/matrix_logic/ -q(156 passed)MiniMaxAI/MiniMax-M3-MXFP8staged successfully in node-local/raid/hf-hub-cache(~414 GB)Hardware Validation
Single-job GHA smoke: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/27480947060
The TP8, EP1, 1k1k, concurrency-4 smoke completed successfully on
chi-mi300x-054, including result processing and artifact upload:The model loaded on all 8 MI300X GPUs, the vLLM API became healthy, all requests completed successfully, and the benchmark artifact was uploaded. The earlier Slurm, cache-directory, and rootless GPU-device permission blockers are resolved.
The slow throughput is expected for this image on MI300X: vLLM reports no native MXFP8 MoE backend on gfx942, so the MoE runs through the BF16 emulation path with eager execution.
Accuracy Validation
The original full sweep exposed a real ROCm FP8-attention accuracy failure: vLLM warned that MiniMax-M3-MXFP8 has no calibrated q/prob scales and fell back to scale 1.0. GSM8K scored 0.0099 strict and 0.0296 flexible, with degenerate non-terminating outputs.
The MI300X launcher now keeps the default BF16 KV cache. Direct TEP8 compute-node checks answered three known GSM8K prompts correctly. The matching one-job GHA eval passed: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/27485772194
Sweep
Full sweep passed on all 14 H100-aligned throughput points and both GSM8K eval jobs: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/27485974465
Note
Low Risk
Changes are benchmark config, a new launch script, and MI300X Slurm/container plumbing; no production inference or auth paths are touched.
Overview
Adds a day-zero
minimaxm3-fp8-mi300x-vllmmatrix entry for MiniMax-M3 MXFP8 on MI300X using the samevllm/vllm-openai-rocm:minimax-m3image as MI355X, with an H100-aligned fixed-seq sweep (TP8 and TP8+EP8 on 1k1k and 8k1k).Introduces
minimaxm3_fp8_mi300x.sh, which mirrors the MI355X MiniMax-M3 vLLM serve shape (block size 128, prefix caching off, TRITON_ATTN, eager, language-model-only, MiniMax parsers) but omits FP8 KV cache on gfx942 so attention stays on default BF16—documented as avoiding bad accuracy from missing ROCm FP8 q/prob scales.MI300X runner fixes: Slurm
--excludenow uses short node namechi-mi300x-049, and the Enroot container mounts/dev/kfdand/dev/drifor rootless GPU access. Documents the new config inperf-changelog.yaml.Reviewed by Cursor Bugbot for commit c0c47fb. Bugbot is set up for automated code reviews on this repo. Configure here.